Heuristic Evaluation of User Experience – Case Nokia

Along several years, Nokia has utilized user experience heuristics for assessing design prototypes with a panel of internal experts. In this paper, we describe how we have tailored the traditional heuristic evaluation method to suit the needs of a large multinational corporate. HEURISTIC EVALUATION Heuristic evaluation method [3] has traditionally been used for evaluating usability against a set of principles, e.g. the 10 usability heuristics by Jakob Nielsen [4]. Heuristic evaluation of user experience (UX) in a broader sense seems to be rare, which, we believe, is due to the situational nature of UX. It is hard to derive universal UX heuristics that would imply good UX for all kinds of users in various different situations with various kinds of products and services. This is a core reason why UX heuristics need to be tailored for the different evaluation cases. The few examples of UX heuristics have indeed been developed for specific cases: UX of crossplatform social networking systems [7], playful experiences of games [1], and persuasive health technologies [2]. A few years ago, we reported heuristic evaluation as one of the most potential methods in the early phase of product development [6]. The main benefits include • fast application of the method after building the infrastructure for systematic heuristic evaluation • possibility to run the evaluation in different parts of the world simultaneously • possibility to evaluate early ideas that are not articulated very clearly yet • experts can understand the future developments that will influence the UX of futuristic concepts. The main challenges in heuristic evaluation of UX include setting up the infrastructure for systematic and rapid heuristic evaluation, and foreseeing how ordinary users will experience the concept once it comes out. The heuristics should be formulated so that the latter challenge can be alleviated. OVERVIEW OF THE EVALUATION CONTEXT UX123 is an infrastructure established inside Nokia for evaluating the UX design competitiveness of mobile products in a systematic manner. The UX123 team serves product development with a pool of internal experts on different sites of the company. A set of UX heuristics has been developed to make the evaluation systematic. Whenever a new evaluation is needed, a subset of evaluators is recruited to assess the UX of a given user journey on the given products against the heuristics. Competitor products are assessed with the same set of heuristics. The evaluators report their findings in a review workshop and the UX123 team analyses and categorizes the findings and reports to the product team as well as other stakeholders. The evaluation can be done in different phases of product development. Below we describe how the method-resources [8] have been tailored over time in different evaluation cases within Nokia. Participant recruitment The principles of expert recruitment are the same in all evaluation cases: experts are senior UX design professionals with experience from product development both inside and outside of Nokia. Design representation Since heuristic evaluation is conducted in various different phases of product development, the representation of the design that experts evaluate varies accordingly. In the early phase, the representation may be a concept demonstration, in a later phase an actual prototype. Experience has shown that UX heuristics are best applied to concepts that allow or simulate interaction. Concepts without interactive aspects tend to demonstrate aspects such as visual design well, whereas other aspects such as input mechanisms often cannot be assessed. As a result, all of the heuristics may not be applicable and a holistic assessment of the UX may not be possible. The UX123 evaluation process includes competitor evaluations. Naturally, these devices are products on the market.